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NAVTRAEQUIPCEN  IH-250 
SUMMARY 


A considerable  effort  has  been  undertaken  at  the  Naval  Training  Equip- 
ment Center’s  Analysis  and  Design  Branch,  Computer  and  Human  Factors  Labora- 
tories toward  answering  three  questions  regarding  computer  generated  visual 
system  technology.  The  questions  were: 

a.  Dc  visual  system  presentation  delays  on  the  order  of  0.1  seconds 
have  any  adverse  effects  on  pilot  trainee  learning  ability? 

b.  Do  the  presentation  delays  cause  the  pilot  subjects  to  exercise 
their  piloting  skills  differently  than  when  their  visual  stimuli  are  not 
delayed? 


c.  What  is  the  nature  of  the  differences  in  piloting  techniques 
utilized  when  the  pilot's  visual  stimuli  have  been  delayed,  if  any? 

Questions  a,  b,  and  c have  been  answered  by  a two  experiment  study 
for  the  specific  task  of  landing  an  aircraft  simulator,  with  performance 
similar  to  an  F-4,  on  an  aircraft  carrier  visual  display  generated  by 
computer  generated  imagery. 

Experiment  1 of  the  study  addressed  the  first  question  posed.  Twelve 
pilot  subjects  of  varying  age  and  background  were  asked  to  "fly"  carrier 
approaches  both  with  and  without  a 0.1  second  delay  in  the  visual  scene 
presented  to  them.  The  performance  criterion  of  merit  was  the  number  of 
trials  required  for  the  subjects  to  complete  three  successive  carrier 
arrestments . 

Experiment  2 of  the  study  addressed  the  second  and  third  questions 
posed.  For  Part  1 of  Experiment  2,  twelve  pilot  subjects  were  asked  to 
"fly"  carrier  approaches  until  five  successful  carrier  arrestments  were 
made.  Real  time  data  recording  was  used  to  record  six  pilot  control 
inputs.  A statistical  unit  of  measure  known  as  the  variance  was  computed 
for  each  of  the  control  inputs.  These  variances  were  compared  for  the 
delay  and  no-delay  cases  using  some  standard  statistical  analytical 
procedures  known  as  multivariate  analyses. 

Part  2 of  Experiment  2 addressed  question  c and  utilized  the  data 
gathered  under  Part  1.  Fast  Fourier  transforms  were  performed  on  the 
pilot  control  inputs  for  the  delay  and  no-delay  conditions  transforming 
the  seemingly  random  time  histories  to  the  frequency  domain  for  easier 
interpretation.  The  frequency  spectra  for  the  delayed  environment  of 
the  recorded  control  parameters  were  compared  to  those  for  the  non-delayed 
environment. 

The  results  of  this  study  indicated: 

a.  In  Experiment  1,  the  difference  between  the  mean  number  of  trials 
required  by  the  pilot  subjects  to  reach  criterion  performance  in  the  delay 
condition  and  the  mean  number  in  the  no-delay  condition  was  not  statistically 
significant.  In  fact,  except  for  the  earliest  trials,  the  differences 
between  raeau  performance  with  nc  delay  and  mean  performance  with  delay 
were  practically  non-existent. 
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b.  The  pilots  exercised  their  piloting  skills  differently  for  two  of 
the  six  control  parameters  analyzed  in  Experiment  2 of  the  study.  The 
variances  of  the  lateral  control  deflection  and  force  were  significantly 
different  for  the  delayed  presentation  than  in  the  non-delayed  presentation. 
The  probabilities  for  the  results  obtained  to  have  been  caused  by  a random 
occurrence  were  P = .0083  for  the  lateral  control  deflection  change  in 
variance  and  P = .0392  for  the  lateral  control  force  change  in  variance. 

The  differences  in  the  other  four  control  inputs  analyzed  were  not  statis- 
tically significant  (P  not  less  than  .05). 

c.  Differences  in  the  frequency  spectra  for  the  two  delayed  conditions 
of  the  pilot  subject  inputs  were  averaged  over  all  tasks  and  subjects. 

These  differences  in  the  frequency  spectra  represent  the  influence  of  the 
delayed  visual  presentation. 
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SECTION  I 


INTRODUCTION 


Because- of  the  current  national  economy,  the  fuel  shortage,  concern 
for  ecology,  and  the  ever  increasing  complexity  and  cost  of  modern  weapon 
systems,  there  is,  and  will  likely  continue  to  be,  emphasis  on  the  develop- 
ment and  utilization  of  sophisticated  flight  simulators.  Military  and 
commercial  aircraft  users  are  investing  heavily  in  flight  simulators  equipped 
with  visual  systems  and  in  visual  systems  to  be  attached  to  existing  flight 
simulators. 

In  general,  visual  simulators  are  conceived  as  add-on  systems  to  flight 
trainers.  Investigation  of  interfacing  such  systems  has  been,  historically, 
and  typically,  less  than  rigorous.  Addition  of  one  system  to  another  seems 
inevitably  to  affect  the  operation  of  the  combination.  Such  is  the  case 
with  visual  systems  when  attached  to  flight  simulators. 

An  inherent  delay  exists  between  the  time  a visual  system  receives 
its  inputs  and  the  time  a visual  presentation  is  displayed.  For  example, 
the  Computer  Generated  Image  Advanced  Development  Model  visual  system 
attached  to  Device  2F90,  a TA-4J  OFT,  at  Kingsville  Naval  |ir  Station  (NAS), 
Texas,  in  late  1973,  required  a little  in  excess  of  100  msA  to  generate  a 
visual  scene.  This  time  delay  added  to  the  50  ms  update  cycle  time  of  the 
2F90,  represented  | 200  percent  change  in  time  related  effects  on  the  pilot's 
control  responses. 

The  question  naturally  arose  as  to  what  effect  this  additional  delay 
is  likely  to  have  on  the  training  effectiveness  of  a flight  simulator 
system. 
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^Healy,  L.  D.  and  Cooper,  F.  R. , "Verification  of  Simulator  Performance  by 
Frequency  Response  Measurement,"  Proceedings  of  the  6th  NAVTRAEQUIPCEN/ 

* Industry  Conference,  Nov  13-15,  1973,  NAVTRAEQUIPCEN  IH-226. 
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O'Connor,  F.  E.,  CAPT  USN,  Dr.  B.  J.  Schinn,  and  Dr.  W.  M.  Bunker,  "Prospects, 
Problems,  and  Performance:  A Case  Study  of  the  First  Pilot  Trainer  Using 
CGI  Visuals,"  Proceedings  of  the  6th  NAVTRAEQUIPCEN/Industry  Conference, 

Nov  13-15,  ] 973,  NAVTRAEQUIPCEN  IH-226. 
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SECTION  II 

STATEMENT  OF  THE  PROBLEM 


The  purpose  of  the  experiment  was  to  attempt  to  answer  the  following 
questions: 

a.  Does  a 100  ms  delay  of  a visual  presentation  affect  pilot  learning 
performance? 

b.  Do  pilots  perform  their  piloting  skills  differently  when  their 
visual  stimuli  have  been  delayed  for  100  ms? 

c.  If  pilots  do  perform  their  skills  differently  when  visual  stimuli 
are  delayed  100  ms,  in  what  way(s)  is  their  performance  different? 


NAVTRAEQUIPCEN  IH-250 
SECTION  III 

EXPERIMENT  DESCRIPTION 


APPROACH 

The  previous  questions  were  addressed  by  two  experiments.  Experiment  1 
was  designed  to  answer  Question  a.  Experiment  2 was  designed  to  answer 
Questions  b and  c. 

The  approach  taken  to  answer  Question  a was  to  design  specific  carrier 
approach  tasks  which  incorporated  both  delay  and  no-delay  conditions  to  be 
learned  by  pilot  subjects.  The  pilot  subjects  were  then  required  to  fly 
the  tasks.  An  analysis  of  the  number  of  carrier  approach  trials  taken  to 
achieve  an  established  successful  criterion  of  performance  was  then  conducted. 

The  approach  taken  to  answer  Questions  b and  c was  to  focus  the  investi- 
gation on  the  pilot/simulator  interface  — the  flight  controls.  Pilot  control 
displacements  and  forces  were  measured  while  flying  specific  carrier  approach 
tasks  with  and  without  100  milliseconds  (ms)  delay.  An  analysis  of  the  re- 
corded measurements  was  accomplished  to  determine  if  pilots  manipulated  the 
controls  with  more  or  less  displacements  and/or  with  more  or  less  forces 
when  their  visual  stimuli  were  delayed.  Finally,  the  measurements  of  control 
displacements  and  forces  were  subjected  to  a Fourier  analysis  to  examine,  in 
the  frequency  domain,  the  effects  of  the  100  ms  visual  presentation  delay  on 
flight  control  activity. 

HARDWARE  AND  SIMULATION  SOFTWARE  USED 

The  experiments  were  conducted  with  the  Naval  Training  Equipment  Center's 
(NAVTRAEQUIPCEN' s)  TRADEC  F-4  Flight  Simulator.  This  simulator  system  con- 
sists of  a Xerox  Data  System  Sigma  7 digital  computer  with  a full  complement 
of  general  purpose  digital  computer  peripheral  equipment  (figures  1 and  2) , 
a four-degree-of-freedom  motion  platform  (figure  3) , a variable  configuration 
simulated  aircraft  cockpit  (figure  4),  and  an  operator's  control  console 
(figure  5). 

The  computer  system  hardware  consists  of  48,000  words  of  core  storage, 
13.7  million  bytes  of  random  access  disc  memory,  four  magnetic  tape  drives, 
a high-speed  line  printer,  card  reader,  card  punch,  paper  tape  reader/punch, 
and  a Calccmp  incremental  plotter.  The  simulator  software  is  a program 
which  simulates  the  F-4  aircraft.  The  F-4  simulator  is  utilized  in  the  con- 
duct of  research  in  various  aspects  of  simulation  techniques  and  of  human 
factors  relating  to  simulation.  The  program  is  written  to  support  operator's 
console  functions  such  as  establishing  modes  of  flight,  recording  of  data, 
aiding  in  conducting  tests  and  establishing  different  conditions  and  confi- 
gurations of  flight.  The  program  allows  recording  of  up  to  165  selectable 
parameters  on  magnetic  tape  each  program  iteration  cycle,  i.e.,  every  50  ms. 

The  simulation  program  was  modified  to  provide  appropriate  operator 
control  of  the  conduct  of  this  experiment.  Program  modifications  provided 
for: 
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Figure  3.  TRADEC  Four  Degree  of  Freedom  Motion  System 


NAVTRAEvUIPCEN  IK-250 


NAVTRAEQUIPCEN  IH-250 


a.  Operator  insertion  of  test  subject  identification  and  carrier 
approach  task  conditions  (e.g.,  Delay/No-Delay,  Task  Selection,  etc.)* 

b.  Presetting  the  position  of  the  simulated  aircraft  to  one  of  three 
selectable  points  in  space  from  which  carrier  approaches  began. 

c.  Operator  release  of  control  of  the  simulator  to  a pilot  subject 
enabling  him  to  fly  an  approach  and  attempted  arrestment  on  a visually 
depicted  carrier. 

* d.  Appropriate  termination  of  each  carrier  approach. 

e.  Control  of  recording  selected  data  on  magnetic  tape  during  each 
approach. 

The  F-4  simulator  was  interfaced  with  an  Evans  and  Sutherland  Line 
Drawing  System  (LDS)  I line  drcwing  visual  CRT  display  system3  which  provides 
a 19°  horizontal  by  19°  vertical  field  of  view.  This  monochromatic  visual 
system  consists  of  a line  drawing  scope  shown  in  figure  5,  a special  purpose 
high-speed  processor,  figure  6,  and  an  associated  slave  scope  located  in  the 
simulated  cockpit  in  view  of  the  pilot,  shown  in  figure  4.  The  special  pur- 
pose high-speed  processor  accepts  aircraft  and  aircraft  carrier  position  and 
orientation  information  from  the  simulator  computer  and  produces  the  ~orrect 
perspective  picture  at  the  two  display  stations  in  real  time.  The  time  re- 
. , quired  for  the  visual  system  to  compute  and  display  the  aircraft  carrier 

^ scene  used  in  these  experiments  varies  from  12.5  ms  to  25  ms.  The  time 

taken  within  this  range  depends  upon  the  number  of  lines  that  are  in  view 
of  the  pilot's  eyepoim.,  which  is  dependent  upon  the  distance  between  the 
aircraft  and  the  aircraft  carrier  as  the  approach  to  arrestment  progresses. 

The  F-4  simulator  program's  iteration  cycle  is  50  ms.  Position  and 
orientation  of  the  aircraft  and  aircraft  carrier  are  computed  each  program 
iteration.  The  method  of  simulating  100  ms  additional  delay  in  the  visual 
system  was  accomplished  by  withholding,  from  the  Evans  and  Sutherland  visual 
system,  this  aircraft  and  carrier  positional  information  for  two  program 
iteration  cycles  (2  iterations  x 50  ms  per  iteration  = 100  ms).  This  was 
accomplished  by  software,  the  implementation  of  which  is  illustrated  in 
figure  7.  Carrier  and  aircraft  positioning  information  was  stored  in  buffers, 
the  first  buffer  containing  the  position  information  calculated  during  the 
preceding  program  iteration  cycle,  (therefore  50  ms  old),  the  second  buffer 
the  iteration  cycle  before  that  (100  ms  old),  etc.,  with  the  9th  buffer  hold- 
ing the  information  calculated  during  the  9th  previous  iteration  (i.e.,  450 
ms  old). 


3 

Sutherland,  Ivan  E.  and  Dan  Cohen,  "Display  Techniques  for  Simulation," 
Technical  Report:  NAVTRADEVCEN  70-C-0025-1. 
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Figure  6.  Evans  and  Sutherland  Special  Purpose 
High-Speed  Processor.  LDS  I 
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Figure  7.  Time  Delay  Implementation  and 
Signal  Interfacing 
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At  the  end  of  each  simulation  program  iteration  cycl i the  contents  of 
buffers  0 through  9 were  transferred,  or  shifted  to  the  adjacent  buffer. 
Information  in  the  9th  buffer  was  discarded.  Selection  of  a given  buffer 
to  be  presented  to  the  Evans  and  Sutherland  visual  system  therefore  deter- 
mined the  amount  of  visual  system  time  delay  simulated.  The  abject  experi- 
ment utilized  the  selection  of  "buffer"  2 when  a delayed  task  was  to  be  flown 
and  "buffer"  0 when  a no-delay  task  was  to  be  flown. 

The  implementation  just  described  resulted  in  effectively  adding  100  ms 
time  delay  to  the  actual  time  required  by  the  Evans  and  Sutherland  system  to 
produce  and  display  the  position  of  the  aircraft  carrier  scene.  Therefore, 
the  actual  visual  cue  delays  presented  to  the  pilot  subjects  was  12.5  ms  to 
25  ms  for  the  no-delay  condition  and  112.5  ms  to  125  ms  for  the  delayed  con- 
dition. 

PILOT  SUBJECTS  USED 

Sixteen  Navy,  Marine  and  Air  Force  pilots  and  former  pilots,  assigned 
to  or  employed  as  civilians  by  the  NAVTRAEQUIPCEN,  or  employed  and  self- 
employed  in  industry  in  the  Orlando  area,  volunteered  their  time  to  serve 
as  pilot  subjects  in  the  experimentation.  (Table  1 contains  a summary  of 
their  flying  experience.)  All  but  two  were  carrier  qualified  from  two  and 
one-half  years  to  twenty-five  years  ago. 

TASKS  PERFORMED  BY  PILOT  SUBJECTS 

The  tasks  selected  were  rather  exacting  and  purposely  so,  for  it  was 
thought  that  if  an  artificial  delay  of  100  ms  were  to  have  an  effect,  it 
would  show  up  more  readily  in  the  more  difficult  parts  of  the  flight  training 
regimen. 

The  basic  task  for  the  pilot  subject  was  to  learn  to  land  a simulated 
aircraft  on  the  carrier  deck  displayed  on  a Cathode  Ray  Tube  (CRT)  screen. 

Six  variations  of  this  basic  task  were  used.  Two  were  considered  a_  priori 
to  be  of  least  difficulty,  comparatively  speaking,  two  of  moderate  difficulty 
and  two,  the  most  difficult.  This  was  done  in  order  to  afford  the  pilot  sub- 
jects some  early  opportunity  of  success  to  prevent  p ssible  discouragement  on 
their  part  and  also  in  the  later  analysis  to  determine  if  an  interaction  exis- 
ted between  Delay  and  Task  Difficulty. 

Certain  initial  conditions  were  common  to  all  six  task  variation^.  In 
each  case,  the  carrier  moved  at  a rate  of  thirty-five  (35)  knots.  The  air- 
craft was  always  positioned  one  (1)  nautical  mile  from  the  carrier  at  an 
altitude  of  three  hundred  ninety  (390)  feet  and  at  an  airspeed  of  one  hundred 
thirty-five  (135)  knots  (i.e.,  on  the  glide  slope  and  at  the  correct  airspeed). 
Except  for  pilot  control  positions,  initial  conditions  were  the  same  for 
each  approach  trial.  Each  successful  approach  trial  required  about  30 
seconds  flight  from  the  time  the  pilot  subject  was  given  control  until 
approach  termination  occurred. 

The  six  task  variations  were  as  follows: 


Task  A (Least  Difficult) 
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The  aircraft  was  set  600  feet  to  the  light  of  the  center  line  of  the 
carrier's  angle  deck,  figure  8.  The  pilot  subject  was  required  to  Bake  a 
left  turn  to  line  up  on  the  center  line  of  the  carrier's  angle  deck. 

Task  B (Least  Difficult) 

The  aircraft  was  set  directly  on  the  glide  slope  and  on  the  center 
line  of  the  carrier's  angle  deck,  figure  9.  No  turns  were  required  and 
the  pilot  subject's  objective  was  to  hold  the  aircraft  on  the  glide  slope 
until  arrestment. 

Task  C (Moderately  Difficult) 

This  task  was  the  same  as  Task  B,  figure  9,  except  that  an  arbitrary 
level  of  'urbulence  representative  of  "light  turbulence"  flying  conditions 
was  added  to  the  simulator  motion  system. 

Task  D (Moderately  Difficult) 

The  aircraft  was  set  60C  peet  to  the  left  of  the  center  line  of  the 
carrier's  angle  deck,  figure  10.  The  pilot  subject  was  required  to  make 
a right  turn  to  line  up  on  the  angle  deck's  center  line. 

Task  E (Most  Difficult) 

This  was  the  same  as  Task  D,  figure  10,  (right  turn  required  from 
600  feet  to  the  left  of  the  angle  deck  center  line)  with  an  arbitrarily 
selected  more  severe  level  of  turbulence,  representative  of  "heavy  tur- 
bulence" flying  conditions,  added  to  the  simulator  motion  system. 

Task  F (Most  Difficult) 

This  was  the  same  as  Task  A,  figure  8,  (left  turn  required  from  600 
feet  to  the  right  of  the  angle  deck  center  line)  with  the  more  severe  level 
of  turbulence  added  to  the  problem. 

There  were  five  conditions  which  had  to  be  met  in  order  for  a trap 
(aircraft  arrestment)  to  be  successful: 

a.  The  trap  area  on  the  carrier  deck  was  rectangular  in  shape  and 
simulated  a carrier  deck  area  50  feet  wide  by  80  feet  long.  A trap  was 
possible  if  the  aircraft  center  of  gravity  was  in  an  altitude  range  of 
64  to  69  feet  above  sea  level  and  within  the  trap  ai^a. 

b.  The  landing  gear  had  to  be  down. 

c.  The  rate  of  descent  of  the  aircraft  had  to  be  less  than  or  equal 
to  1000  feet  per  minute  as  it  entered  the  space  defined  in  paragraph  a. 
above . 

d.  The  airciaft  could  not  be  pitched  down  more  than  two  degrees  from 
horizontal  and  not  be  pitched  up  more  than  eighteen  degrees  from  horizontal 
as  it  entered  the  space  defined  in  paragraph  a. 
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Figure  9.  Visual  Display  Starting  Position  (renter) 
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Figure  10.  Visual  Display  Starting  Position  (Right) 
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e.  The  aircraft  could  not  be  rolled  to  the  left  or  right  mo*  than 
fifteen  degrees  from  horizontal  as  it  entered  the  space  defined  in  para- 
graph a. 

DESCRIPTION  OF  EXPERIMENTS 

Two  experiments  were  designed  to  answer  the  questions  stated  in  Section 
II,  Statement  of  the  Problem.  The  first  experiment  was  designed  to  address 
Question  a of  Section  II.  The  second  experiment  was  designed  to  address 
Questions  b and  c of  Section  II. 

The  procedure  common  to  both  experiments  was  as  follows.  Each  pilot 
subject  was  briefed  before  entering  the  simulator  cockpit.  After  the 
briefing  and  while  the  pilot  subject  was  buckling  into  the  cockpit  seat, 
the  operator  entered  the  pilot's  identification  code,  task  selection,  and 
delay /no-delay  control  code  into  the  simulator  program.  The  operator  then 
preset  the  simulated  aircraft's  position  to  a point  in  space  associated 
with  the  selected  task.  The  pilot  at  this  time  could  see  a visual  display 
of  an  aircraft  carrier  as  seen  from  390  feet  altitude,  at  a distance  of 
one  mile,  and  either  600  feet  left  of,  600  feet  right  of,  or  directly 
aligned  with  the  center  line  of  the  carrier's  angle  deck,  figures  10,  9, 
and  8 respectively.  The  simulated  flight  airspeed  was  set  at  135  knots. 

When  the  pilot  subject  indicated  he  was  ready,  the  operator  released  con- 
trol of  the  simulator  to  the  pilot.  The  pilot  was  then  completely  in 
control  of  the  flight  simulator.  Recording  of  the  pilot's  flight  control 
activity  on  magnetic  tape  began  at  the  instant  the  operator  released  control 
to  the  pilot.  The  pilot  was  then  required  to  fly  the  approach  visually  to 
the  displayed  carrier  and  attempt  an  arrestment.  Automatic  data  recording 
every  50  ms  on  magnetic  tape  continued  until  the  approach  terminated  with 
an  arrestment,  a bolter,  a wave  off,  or  a crash.  Upon  conclusion  of  the 
approach,  the  operator  reset  starting  conditions  as  described  previously 
so  that  the  pilot  could  attempt  another  approach.  The  pilot  subject  con- 
tinued making  approaches  in  this  fashion  until  successfully  completing  the 
established  success  criterion  for  the  experiment.  After  successfully 
completing  a task,  the  operator  inserted  appropriate  task  selection  and 
delay  codes  into  the  program  to  set  up  the  subsequent  task. 

EXPERIMENT  1 

PROCEDURE.  Twelve  of  the  pilot  subjects  practiced  each  task  until  each 
was  proficient  in  task  performance.  A pilot  subject  was  considered  to 
have  learned  a task  if  he  made  three  successful  arrestments  in  a row  in 
that  task.  The  dependent  variable  was  number  of  trials  to  '..riterion  per- 
formance for  each  task. 

The  tasks  were  always  presented  in  the  a.  priori  oHcr  of  difficulty, 
that  is.  Tasks  A and  G preceded  Tasks  C and  D ana  the  latter  preceded  Tasks 
E and  F.  Within  this  general  order,  however,  the  Delay  vs  No-Delay  condi- 
tion was  interleaved  so  that  one  condition  may  not  have  an  obvious  advantage 
over  the  other  due  to  "practice  effects."  The  order  in  which  the  pilot 
subjects  learned  the  tasks  is  summarized  in  table  2.  Each  pilot  subject 
was  assigned  to  a presentation  order  at  random  with  the  restriction  that 
the  last  pilot  subjects  were  assigned  to  orders  to  maintain  the  overall 
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Table  2.  ORDER  OF  TASK  PRESENTATION 
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balance  in  table  2.  The  numbers  in  the  body  of  table  2 specify  the  order 
in  which  each  pilot  subject  learned  the  tasks  under  the  two  (Delay /No-Delay) 
conditions. 

Table  2 indicates  that  six  pilot  subjects  learned  Task  B first,  three 
in  the  Delay  condition  and  three  in  the  No-Delay  condition.  Those  three 
that  had  learned  Task  B in  the  No  Delay  condition  then  learned  Task  A in 
the  Delay  condition.  Those  three  that  had  learned  Task  B in  the  Delay 
condition  then  learned  Task  A in  the  No-Delay  condition.  The  other  six 
pilot  subjects  learned  Task  A first,  three  with  No-Delay  and  three  with 
Delay,  and  then  learned  Task  B second  with  the  conditions  reversed.  Tasks 
C and  D,  and  then  E and  F were  learned  in  the  orders  indicated  in  table  2. 
The  pilot  subjects  were  not  informed  of  the  Delay  or  No-Delay  conditions. 

Overall  then,  each  of  the  twelve  pilot  subjects  learned  six  tasks, 
two  tasks  at  each  of  the  three  Difficulty  levels,  and  at  each  Difficulty 
level,  one  under  the  No-Delay  condition  and  one  under  the  Delay  condition. 
Each  pilot  subject  was  considered  to  have  learned  each  task  when  he  per- 
formed three  successful  entrapments  in  a row  (successful  performance). 

The  dependent  variable  was  the  number  of  trials  on  each  task  required  to 
reach  successful  performance. 

DATA  RECORDED.  A log  was  kept  of  each  pilot  subject's  carrier  approach 
trials  for  each  task  and  each  delay  condition.  The  log  contained  the 
results  of  each  approach,  i.e.,  trap,  bolter,  wave  off,  or  crash.  Figure 
11  is  a sample  of  the  log. 

The  date,  pilot  identification  code,  and  task  sequence  designation 
were  recorded  on  each  page  of  a subject's  record.  The  approach  trial 
number,  the  approach  outcome  (e.g.,  wave  off,  bolter,  crash,  or  trap), 
the  number  of  wire  caught  (wire  1 through  4) , the  task  designation  (tasks 
A through  F,  with  indication  of  delay  or  no-delay),  and  remarks,  as 
applicable,  were  recorded  for  each  approach  trial.  The  remarks  column  was 
intended  primarily  to  note  spontaneous,  off-hand  comments  from  the  subject 
pilot  that  may  have  supported,  or  been  relevant  to,  the  analysis  of  the 
experiment. 

As  indicated  in  figure  11,  a task  was  flown  until  the  subject 
achieved  three  successive  traps.  The  next  task  called  for  in  the  given 
pilot's  task  sequence  was  then  set  up.  The  subject  continued  in  this 
fashion  until  completing  all  six  tasks. 

DATA  ANALYSIS  AND  RESULTS.  The  analysis  of  variance  model  used  in  the 
data  analysis  is  a special  case  of  three-way  classification  mixed  model 
in  which  the  Delay/No-Delay  condition  and  the  Task  conditions  are  fixed  ^ 
constants  and  the  assignment  of  the  pilot  subjects  was  a random  variable. 


4 

See  McNamar,  Quinn:  Psychological  Statistics,  John  Wiley  § Sons,  N.Y.,  N.Y., 

1969,  pp  364-371 
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The  results  of  the  analysis  of  variance  are  presented  in  table  3.  The 
main  interest  was  testing  for  effects  of  the  two  nanipulated  variables  (Delay/ 
No-Delay  and  Task  Difficulty)  and  their  interaction.  For  the  influence  of  the 
Delay /No-Delay  condition  on  pilot  subject  performance,  F = 0.53  which  is  ob- 
viously not  significant.  For  the  effect  of  Task  Difficulty,  F = 6.666  which 
is  a statistically  significant  ratio  (.05>p>  .01,  df  = 2.22)>5  The  Delay  by 
Task  interaction,  F = 0.89,  is  also  not  a significant  result.  No  further 
tests  are  available  in  this  model. 

The  fact  that  task  condition  has  a significant  effect  on  pilot  subjects' 
lfeaming  performance  is  not  surprising.  Rocs.  1 1 that  the  Tasks  were  presented 
roughly  in  the  order  of  difficulty  that  was  agreed  upon  a_  priori.  Thus,  two 
factors  influenced  the  pilot  subjects'  learning  performance  from  task  to  task 
throughout  the  experiment.  The  first  factor  (task  difficulty,  presented  in 
the  order  - relatively  easy  to  difficult)  tended  to  cause  a greater  number 
of  trials-to-criterion  to  be  required  for  the  more  difficult  task.  The 
second  factor,  practice  effect,  operated  in  the  opposite  direction  and  tended 
to  cause  fewer  trials-to-criterion  as  time  went  by  after  longer  practice.  The 
effect  of  the  first  factor,  difficulty  (perhaps  because  the  range  was  narrow), 
was  overshadowed  by  the  effect  of  the  second  factor,  practice,  and  the  gen- 
eral diminution  of  the  trials-to-criterion  on  the  latter  tasks  is  evidenced 
by  the  significance  of  the  Task  factor  in  the  analysis  of  variance. 

Further  evidence  on  this  point  is  presented  in  table  4.  Each  average 
in  table  4 is  based  on  the  performance  of  twelve  pilots.  The  diminution 
of  the  average  number  of  trials-to-criterion  is  especially  noticed  in  pro- 
gression from  the  least  difficult  to  the  moderately  difficult  tasks.  Per- 
formance levels  off  thereafter  so  that  the  "most  difficult."  tasks  were 
learned  in  approximately  the  same  number  of  trials  as  were  the  "moderately 
difficult ." 

Within  each  "Difficulty"  level,  however,  the  differences  between 
the  Delay  and  the  No-Delay  conditions  are  of  no  statistical  nor  practical 
significance.  The  only  possible  exception  from  the  practical  point  of 
view  lies  in  the  "Least  Difficult"  task  level  where  the  average  number  of 
trials-to-criterion  was  greater  under  the  Delay  conditions.  This  difference 
was  due  solely  to  the  performance  of  one  pilot  who  took  157  trials-to- 
criterion  in  the  Delay  condition  (his  first  tcsi)  and  then  made  only  one 
subsequent  error  during  the  remainder  of  the  experiment. 

The  main  conclusion  from  this  part  of  the  study  is  that,  overall,  the 
introduction  of  a 100  ms  df>lay  in  presentation  of  the  visual  information 
had  no  effect  on  the  learning  by  the  pilot  subjects. 


'’For  the  choice  of  the  error  terms  in  these  tests  see  McNamar,  Quinn, 
ibid.,  pp  377-378. 
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Table  3.  ANALYSIS  OF  VARIANCE  FOR  PERFORMANCE 

SCORES  FOR  12  PILOT-SUBJECTS  FOR  TWO  "DELAY" 
CONDITIONS  AND  THREE  LEVELS  OF  TASK  DIFFICULTY 


G 


SOURCE 

SUM 

OF 

SQUARES 

DEGREES 

OF 

FREEDOM 

VARIANCE 

ESTIMATE 

F RATIO 

Delay  (D) 

234.72 

1 

234.72 

0.53 

Task  (T) 

4,649.69 

2 

2,324.85 

6.66 

Subject 

4,890.94 

11 

444.63 

Interaction 

D X T 

852.03 

2 

426.02 

0.89 

D X S 

4,884.95 

11 

444.09 

T X S 

7,684.98 

22 

349.32 

D X T X S 

10,537.30 

22 

478.97 

TOTAL 

33,734.61 

71 

Table  4.  MEAN  NUMBER  OF  TRIALS  TO  CRITERION 


TASKS  A5B 
LEAST 
DIFFICULT 

TASKS  CSD 
MODERATELY 
DIFFICULT 

TASKS  E5F 
MOST 

DIFFICULT 

No  Delay 

20,8 

11.0 

11.2 

Delay 

34.2 

9.4 

10.3 

Both 

27.5 

10.2 

10.7 
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EXPERIMENT  2 

PROCEDURE.  All  pilot  subjects  were  quite  proficient  after  completion  of 
Experiment  1.  The  flight  tasks  and  operating  procedures  were  familiar  to 
them  at  the  beginning  of  Experiment  2. 

The  object  of  Experiment  2 was  to  record,  for  later  analysis,  each 
pilot  subject's  flight  control  activity  while  flying  assigned  carrier 
approaches  with  and  without  the  delay  condition.  Twelve  pilots  completed 
these  tasks.  Task  B,  an  easy  task.  Task  D,  a moderately  difficult  task, 
and  Task  F,  a difficult  task  used  in  Experiment  1 were  chosen  for  use  in 
Experiment  2. 

It  was  found  to  be  convenient  to  refer  to  Tasks  D,  B and  F as  Left, 
Center  and  Right  Tasks,  respectively,  each  with  Delay  (D)  and  with  No-Delay 
(ND).  Subsequent  references  to  tasks  will  be  made  in  this  manner. 

Experiment  2 required  each  pilot  subject  to  make  five  successful 
arrestments  for  each  of  the  Left,  Center,  and  Right  Tasks  with  and  without 
the  delay  condition.  This  resulted  in  a total  of  30  successful  arrestments 
required  of  each  pilot  subject.  Successive  arrestments  were  not  required. 
Typically,  a subject  would  make  40  to  60  approach  attempts  in  achieving  30 
successful  traps.  The  pilot's  control  activity  was  recorded  on  magnetic 
tape  during  all  of  his  approaches,  however,  only  that  recorded  during  suc- 
cessful approaches,  i.e.,  resulting  in  arrestment,  were  subjected  to  later 
analysis.  The  sequence  of  tasks  flown  by  each  pilot  subject  was  identical. 
The  sequence  was  as  follows: 

(1)  C (D)  - Center  with  Delay 

(2)  L (ND)  - Left  with  No-Delay 

(3)  R (ND)  - Right  with  No-Delay 

(4)  L (D)  - Left  with  Delay 

(5)  R (D)  - Right  with  Delay 

(6)  C (ND)  - Center  with  No-Delay 

DATA  RECORDED.  Six  (6)  pilot  control  parameters  were  recorded  on  magnetic 
tape  each  program  cycle.  These  are: 

DDS  - Stabilator  Control  Stick  Deflection 

DSA  - Aileron  Control  Stick  Deflection 

DRP  - Rudder  Pedal  Deflection 

FSSA  - Force  Applied  to  Stabilator  Control  Stick 

FSAA  - Force  Applied  to  Aileron  Control  Stick 

FRPA  - Force  Applied  to  Rudder  Pedal 
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The  three  parts  of  figure  12  are  time  histories  of  the  six  parameters 
recorded  during  an  approach  by  one  of  the  pilot  subjects.  These  plots  are 
typical  of  rll  approaches  made  by  all  pilot  subjects. 

DATA  ANALYSIS-  VARIANCES  IN  CONTROL  FORCES  AND  DISPLACEMENTS.  The  first 
step  in  the  analysis  of  variance  was  to  compute  the  means  and  variances  of 
each  of  the  six  control  parameters  (DSS,  FSSA,  DSA,  FSAA,  DRP,  FRPA) 
recorded  for  each  successful  carrier  approach  made  during  the  experiment. 

The  results  of  these  calculations  were  recorded  on  magnetic  tape  and  were 
listed. 

The  next  step  in  the  analysis  of  Experiment  2 data  was  to  average  the 
five  values  of  variance  for  the  five  successful  approaches  of  a given  task 
(Left,  Center  or  Right,  Delay  or  No-Delay)  for  each  of  the  six  control 
parameters  (DSS,  FSSA,  DSA  ...,  FRPA).  To  aid  in  explaining  this  and  sub- 
sequent steps  in  the  process  followed  in  analyzing  the  data,  consider  the 
three  dimensional  model  shown  in  figure  13.  Figure  13  is  a sample  model 
structure  of  one  of  a typical  recorded  control  parameter.  Each  cell  indi- 
cated on  the  model  represents  the  average  variance  of  the  given  control 
parameter  taken  over  five  successful  approaches  by  one  of  twelve  pilot 
subjects,  flying  one  of  three  basic  approach  tasks  (Left,  Center,  Right) 
with  one  of  tt*o  visual  presentation  time  delay  conditions  (Delay  or  No-Delay). 
For  example,  the  upper  left-hand  cell  entry  shown  on  figure  13  represents 
the  average  of  the  variances  in  a variable  for  five  successful  approaches 
made  by  one  pilot  subject  for  the  left  task  with  delayed  visual  presentation. 
Table  5 contains  the  computed  average  variance  values  for  each  of  the  six 
control  variables  (DSS,  FSSA,  ...,  FRPA)  for  each  pilot  subject  (12  pilots) 
for  each  task  (Left,  Center,  and  Right)  for  the  two  delay  conditions  (Delay 
or  No-Delay). 

At  this  point,  it  is  important  to  draw  attention  to  what  may  be 
subtle  enough  to  confuse.  Note  that  the  analysis  discussed  in  the  remainder 
of  this  section  is  an  analysis  of  variance  in  variances. 

An  Average  of  Statistics  program,  figure  14  (4  parts),  calculated  an 
average  variance  for  each  cell  of  the  model.  The  same  program  was  used' 
to  compute  the  average  variance  of  the  variances  of  each  control  parameter 
for  all  pilot  subjects  in  each  of  the  three  tasks  (Left,  Center,  Right) 
with  and  without  delay.  The  results  are  summarized  in  figures  15,  16,  arid 
17. 


U 


Since  the  entries  in  cells  are  the  average  variance  in  the  control 
parameters  for  a specific  pilot  and  flight  condition,  the  differences  in 
these  entries  represent  the  effect  of  the  flight  conditions  on  the  manner 
in  which  pilots  exercise  their  piloting  skills. 

The  results  of  averaging  the  cell  entries,  shown  in  figure  13,  over 
all  pilot  subjects  are  given  in  tables  6,  7,  and  8,  and  are  plotted  in 
figures  15,  16,  and  17.  Notice  that  for  all  three  starting  positions  the 
delayed  visual  task  had  greater  variance  than  the  non-delayed  for  the 
following  parameters:  longitudinal  control  deflection  (DSS),  lateral 

control  deflection  (DSA)  and  lateral  control  force  (FSAA).  In  fact,  in 
only  four  of  the  eighteen  comparisons  of  variance  (Left,  Center  and  Right 
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TABLE  5 


F I l 6 T SLBwECT  AVERAGE  VARIANCE 

by  Task,  celay  and 
C0MSBL  PARAMETERS 


FIL3T 

celay 

*R  AVE^A 

GE  VARIANCE 

PARAMETER 

C8CF 

ns  delay  left 

center 

RIGHT 

A AAA 

r 

1 .79K49 

•47CC4 

1*02865 

DSS 

AAAA 

N C 

• 733? 1 

• 74278 

•6S76S 

DSS 

DCCt 

C 

1 • 35 1 a*  2 

»6l 5 1C 

1*40882 

DSS 

c^cc 

NC 

1.32648 

1 «349C6 

1*87705 

DSS 

eeee 

C 

2 • C 2 1 ? 9 

• 565  It 

3*40512 

DSS 

eeee 

N C 

1.75525 

• 7 4 0 5 8 

2 *07053 

DSS 

1C  1C 

c 

1 • 29 ~6f 

• 633  3c 

1*31826 

DSS 

1C1C 

NC 

1.C5656 

*7622c 

1*44891 

DSS 

2l'2C 

C 

1 .24214 

1*13251 

1 *51t>36 

OSS 

2C  2C 

NC 

• 7758C 

•71921 

?* 29756 

DSS 

2333 

c 

1.12134 

.7375? 

l*6C4bC 

DSS 

2333 

NC 

1.04*82 

.45345 

l*0S79c 

DSS 

2121 

D 

1.34445 

1«U777 

1*46078 

DSS 

2121 

NC 

1.2678C 

1* 18525 

2*22694 

DSS 

2C3C 

D 

.95905 

. 1 5 5 2 

1*41465 

DSS 

2C3C 

NC 

.85*52 

. -J759* 

1 • 3 8 6 2 4 

DSS 

4M4 

c 

1.56314 

3 • 15>C2? 

2 • C 7 8 £ 5 

CSS 

4444 

NC 

2.26'"'fc5 

•50685 

2*49766 

DSS 

5^55 

fN 

2.14A»4 

• « 2 C f * 

•57841 

DSS 

555c 

NC 

1.65141 

* ^8044 

1*16494 

CSS 

6^66 

c 

1 . 3 6 A 3 C 

2*19703 

2*69?4s 

DSS 

6*66 

NC 

1.276a? 

2*^591 

2*24010 

CSS 

SS95 

c 

.68^24 

•7672a 

•e8(8t 

DSS 

5SSS 

NC 

] .12*78 

1 *56631 

*50822 

DSS 

a4aa 

r* 

.58797 

*68 06 2 

1*91662 

fssa 

AaAA 

NC 

.75771 

•94036 

2 * C 3 8 S 7 

fssa 

C3CC 

r 

.6l77k 

* 4 fa  '5  c ? 

*98656 

fssa 

c^cc 

NC 

•65772 

*43657 

1*42301 

FSSA 

eeee 

c 

1.CS6F5 

*S58?4 

3* 15772 

fssa 

eeee 

NC 

1 .3lr4« 

• 5817f- 

2*33103 

FSSA 
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TABLE  5 (CBKT) 

F1L8T  SLBvECT  AVERAGE  VARIANCE 
by  task,  delay  and 
C9NTR6L  parameters 


PlLQT 

C0De 

CELAY  9fi  AVERAGE  VARIANCE 

N0  delay  left  center  right 

parameter 

1C1C 

D 

.62122 

•51924 

1*38129 

FSSA 

1C1C 

NC 

•6195C 

•463C3 

1*54758 

FSSA 

2C2C 

C 

.85  '58 

1*49083 

1*38756 

FSSA 

2C2C 

NC 

.71787 

*58641 

3.C4751 

FSSA 

2121 

C 

.62551 

.34999 

•77852 

FSSA 

2121 

NC 

.546SC 

*27284 

•91618 

FSSA 

3C3C 

C 

.82738 

.71748 

1*77233 

FSSA 

3C3c 

NC 

.81291 

.46073 

1*87121 

FSSA 

3333 

C 

.88782 

*66710 

1.24463 

FSSA 

3333 

NC 

•73e36 

«35«SC 

1*02932 

FSSA 

4444 

C 

.93118 

2.C1811 

1 *39633 

FSSA 

4444 

NC 

1.64432 

.44698 

2.26562 

F SSA 

5^55 

C 

1.1617C 

*66272 

l»C694g 

FSSA 

5S55 

NC 

.84389 

•53764 

1*19631 

FSSA 

6-66 

c 

1 • C7r  4 4 

1 • 36543 

2.19842 

FSSA 

6666 

NC 

.75133 

1*27535 

2 » 45267 

FSSA 

S39S 

c 

.59769 

•76064 

1 • 1 3 1 6 S 

FSSA 

S - -S 

NC 

.66211 

• 535 1 5 

•56243 

FSSA 

A AA  A 

f> 

u 

1 .97776 

•9o354 

1.93982 

OS  A 

AaAA 

NC 

8.35853 

• 18444 

1.8461C 

CSA 

CCGC 

C 

1 .55881 

• 255c  1 

1*77968 

OSA 

CL'CC 

NC 

1 • 46 C 45 

•08337 

1*24926 

CSA 

eeee 

C 

1.34147 

• 25545 

r *C8  146 

CSA 

eeee 

NC 

1.323C1 

•C94C9 

1 * 4C1 5£ 

DSA 

1C  1C 

C 

R.C8C5C 

• C 8 5 C 1 

3 * 4C622 

OSA 

1 - 1C 

NC 

1.6CT34 

• 32 1 Cu 

2.512C6 

nS  A 

2 -2C 

c 

1 .64234 

• 3255* 

1 *54367 

CSA 

2C2C 

NC 

.87779 

•17638 

1*79266 

DSA 

2121 

c 

1.47654 

• CM  6 1 

• 79037 

DSA 

c 1 2 1 

NC 

1.13744 

•C77C5 

.59372 

DSA 
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TABLE  5 (CQnT) 

filbt  swBwECT  average  variance 

PV  TASK/  CELAY  AND 
CPNTR6L  PARANETtRS 


F I L3T 
CSCE 

CELAY  9R  AVERA 

N6  CFL AV  LEFT 

GE  VARIANCE 
CENTER  RIGHT 

parameter 

3C3C 

C 

• 614C6 

•124C7 

1*11321 

DSA 

3C3C 

NC 

.4=126 

•C76C5 

•9974C 

DSA 

-333 

n 

V 

2.56739 

•57257 

4.36092 

DSA 

3333 

NC 

2.63645 

• 102 1 9 

2*60337 

DSA 

4^44 

C 

. fllC62 

•15485 

1»C653S 

DSA 

4444 

NC 

.77925 

•10321 

•9303c 

DSA 

5^55 

C 

P.52754 

•4*891 

2 *98728 

r'SA 

5555 

NC 

?.7?4?4 

•3o5»3 

2*24545 

>A 

6666 

D 

1.47121 

•55691 

1*83386 

DSA 

6666 

AC 

1.24R65 

•56519 

2» 19428 

DSA 

9999 

c 

1 .27357 

•2336C 

1.12737 

DSA 

S5S5 

NC 

.77931 

•24390 

•73552 

DSA 

aaaa 

C 

. C>6 32 

•02950 

•10315 

FSAA 

A AAA 

NC 

.1147= 

•C1565 

•11719 

FSAA 

C’-'CC 

c 

.04998 

• C 1294 

•07455 

FSAA 

cocc 

NC 

•C5l 23 

•00366 

. o6C6£ 

FSAA 

EEEE 

c 

.C5434 

•CC916 

•10348 

FSAA 

EEES 

NC 

.05440 

•C0329 

•0817C 

FSAA 

1C  1C 

c 

.C9776 

•00565 

•16201 

FSAA 

1C1C 

NC 

• C 8 C 4 3 

•C2168 

•13331 

FSAA 

cC2C 

c 

.C6A35 

• C 1 4 1C 

*06051 

FSAA 

cC2C 

NC 

.C3337 

•CC669 

•09268 

FSAA 

2121 

n 

•C7135 

*00498 

•0^314 

FSAA 

2121 

NC 

•C6596 

•CC313 

•C34C7 

FSAA 

3C3C 

c 

.C15CC 

•00560 

•04682 

FSAA 

3C3C 

NC 

.C14SC 

•CG338 

•04652 

FSAA 

3333 

D 

• 1373S 

•04865 

•20382 

FSAA 

3333 

NC 

.12144 

•01173 

•1271C 

FSAA 

4444 

c 

.02514 

•CC899 

•04942 

FSAA 

4444 

NC 

. C2625 

•CC374 

•04477 

FSAA 
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TABLE  5 < C8M ) 

filst  siBwect  average  variance 
py  task,  dflay  and 
C9NTR6L  PARAMETERS 


FlL0T  CelAY  OR  AVERAGE  VARIANCE  parameter 


C^Cp 

N9 

CELAY  LEFT 

CENTER 

R IG(-*T 

555c 

C 

.13377 

•C3G65 

• l585g 

FSAA 

5555 

NC 

.119AC 

•C2252 

*1228£ 

FSAA 

6666 

c 

•C5965 

•C2S91 

•1C222 

FSAA 

6666 

NC 

.C569E 

•C38C1 

•11338 

ESAA 

SS9S 

C 

•C7?1C 

•CiRec 

•C7731 

FSAA 

c39s 

NC 

•CB316 

•Cl7«6 

• C636£ 

FSAA 

aaaa 

c 

•CCA66 

•CC13? 

•CC327 

pop 

aaaa 

\c 

• CC27  1 

•CC1C7 

•CC33g 

DRP 

CCCC 

c 

•CCf 3C 

•C0CA7 

•CCCA2 

DRP 

COCC 

NC 

•CCCAC 

•CCC3C 

•GCOAC 

DRP 

eeee 

c 

•CC722 

•CC263 

•C1CC3 

DPP 

eeee 

NC 

.Cl  IAS 

•C0C49 

•ClC8C 

DPP 

1C  1C 

r 

.C53SA 

• CO  1 2 1 

•C6762 

DRP 

1C  lc 

NC 

.CA67S 

•C06A3 

•0^623 

DRP 

2 CSC 

P 

.C31P2 

•CC1 20 

•05038 

DRP 

cC2c 

NC 

• j 1 1 1 9 

• 1 j C 1 6 J 

•01839 

DPP 

2:21 

r. 

• L A 7 E 2 

• CO  1 23 

•01249 

DRP 

2121 

NC 

*C7pC5 

•CC217 

•C21 13 

DRP 

3C3C 

r 

•C’lAS 

•CC63S 

•03259 

DPP 

3C3C 

NC 

»C2cl9 

•C0377 

•04126 

DPP 

2 J 3 3 

p 

•cc 1 « 1 

•CCC21 

•CC157 

DRP 

3-33 

NC 

•CCC97 

•CC1C1 

•01463 

DRP 

i>HAH 

r 

L‘ 

*CC^7R 

•C0CS2 

•C0558 

DRP 

A**  Ah 

NC 

.CC*3C 

•CCC51 

•C12C7 

DRP 

c»kr 

r 

• CCR6  1 

•C0C35 

•CC128 

DRP 

kckk 

NC 

•CC16 1 

•C0C7C 

•CC173 

DRP 
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c 

• C A 3 A A 

•CQ190 

• 05229 

DRP 

6666 

NC 

.C2E1C 

•C2A37 

•C451C 
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SS9S 

c 

.07378 

•C027C 

• C25H 

DRP 
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NC 

•C375C 

•CllSC 

•03298 
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TAPLF.  5 (C9NT) 

f i lot  sue*ECT  average  variance 

PV  TA ?K,  DELAV  AND 

central  parameters 


FlLeT 

c8ce 

cflam  *r  average  variance 
N6  DELAY  left  CENTER  RIGHT 

parameter 
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c 

1.85349 

1.5253? 

3.777SC 

FRPA 

a4aa 

NC 

1.C535* 

2*l3C49 
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r> 

L 

.3913? 

•29441 
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NC 
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•31726 
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C 
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•76C66 
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NC 
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3*13841 
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1C  1C 

D 

5.CC461 
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1C  1C 

NC 
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5 *40544 
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cC2C 

C 

3.27354 

•83384 

4.59249 

FRPA 

2-2C 

N C 

1.7952? 

• 86 1 44 

4.21057 

FRPA 

2121 

c 

3.81*90 

• 93255 

5 * 4 1 6C6 

FPPA 

2121 

NC 

$.284?4 

•93558 

12*36712 

FRPA 

*C3C 

C 

8. 98165 

5*2042^ 

18*69986 

FRPA 

3- 3C 

NC 

1C.17°C6 

2*56332 

30. 16136 

FRPA 

2333 

C 

1.53-74 

• 3 5 5 9 c 

1*26641 

FRPA 

3333 

NC 

.47145 

•7326 U 

2*50758 

FRPA 

4444 

c 

1.45613 

.4664!' 

1.75607 

FRPA 

4444 

NC 

1 .41689 

•4®263 

4.77768 

FRPA 

5255 

r* 

L> 

.44555 

• 244 1 8 

*74423 

FRPA 

w v ^ W 

NC 

•*C*73 

•29256 

•4775c 

FRPA 

6666 

c 

s .CCr43 

1.43678 

19.17815 

FRPA 

6*66 

NC 

.51654 

12*32750 

11*41264 

FRPA 

S$S3 

c 

18.68351 

1 .432C5 

11.75544 

FRPA 

SSS5 

NC 

5.75441 

7 *66120 

12.5026C 
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Figure  14,  Flow  Diagram  of  Experiment  Part  2 
Data  Processing 
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ANOVAS  Program 

Analysis  of  variance  of  variances  in 
parameters  (DSS,  FSSA,  DSA,  FSAA,  DRP, 
FRPA)  due  to  delay/no  delay  for  all 
subjects  for  each  separate  task  (left, 
center,  and  right). 

A X S Analysis  of  variance 
A Delay  or  no  Delay 

S Pilot  Subjects 


TASK 
D ND 
A X S 
LIST 


ANOVABS  Program 

Analysis  of  variance  of  variances  in 
parameters  (DSS,  FSSA,  DSA,  FSAA,  DRP, 
FRPA)  due  to  delay/no  delay  and  due  to 
task  (left,  center,  right)  differences 
for  all  pilot  subjects. 

A X B X S Analysis  of  variance 
A Left,  Center,  Right  task 

B Delay  or  No  Delay 

S Pilot  Subjects 


A X B X S 
D ND 
LIST 


Flow  Diagram  of  Experiment  Part  2 
Data  Processing  (CONI) 
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AVERAGES 

Compute  averages  of  control  parameters 
and  subtract  from  time  histories  to 
form  modified  time  histories. 


AUGMENT 

Augment  modified  time  histories  out  to 
1024  data  points  for  all  runs  with 
zeros  for  DFFT  processing. 


Compute  DFFT's  for  all  modified  time 
histories. 
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AVERAGE 

Average  DFFT's  for  each  harmonic 
for  each  subject  over  his  5 runs 
for  a particular  task. 


Verifica- 

tion 

listing 
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Flow  Diagram  of  Experiment  Part  2 
Data  Processing  (CuNT) 
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Table  6.  STATISTICAL  SUMMARIES  LONGITUDINAL 
AVERAGE  VARIANCES-! 2 SUBJECTS 


DISPLACEMENT  (DSS) 


LEFT 

CENTER 

RIGHT 

ROW 

MEANS 

DELAY 

1.4068 

1.1192 

1.6906 

1.4055  FTask  = 6.2534** 

^Tn5k  - 007? 

NO 

DELAY 

1.2698 

1.0398 

1.6401 

FDel ay  = .8934 

1 *3166  PDel ay  = .3674 

COLUMN 

MEANS 

1.3383 

1.0795 

1.6654 

rTask  X Delay  = -0428 

PTask  X Delay  = .9585 

FOI 

LEFT 

A) 

RIGHT 

ROW 

MEANS 

DELAY 

.8546 

.8666 

1.5983 

1.1065  FTask  = 39.7728** 

^Tn*rlr  - 0000 

mm 

.8395 

.6460 

1.7487 

ia>K  • uuuu 

FDelay  = .1210 

1 '°781  PDelay  = .7332 

c 

m 

.8471 

.7563 

1.6735 

Task  X Delay  = 1.0796 

PTask  X Delay  = .3582 

* STATISTICALLY  SIGNIFICANT  AT  .05  LEVEL 

**  STATISTICALLY  SIGNIFICANT  AT  .01  LEVEL 
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Table  7.  STATISTICAL  SUMMARIES  LATERAL  AVERAGE 
VARIANCES-! 2 SUBJECTS 


DISPLACEMENT  (DSA) 


LEFT 

CENTER 

RIGHT 

ROW 

MEANS 

DELAY 

1.6824 

.3232 

1 .9945 

1.3334  FTask  =40.5225** 

^t.v-  k - nnnn 

NO 

DELAY 

1.4481 

.2015 

1.5918 

lu!>l\  * wUUw 

FDelay  = 10.2748** 

1.0804  PDelay  . >0083 

c 

COLUMN 

MEANS 

1.5652 

.2623 

1.7932 

rTask  X Delay  = 1.8443 
PTask  X Delay  = .1804 

FOf 

LEFT 

ICE  (FSA7 
CENTER 

0 

RIGHT 

ROW 

MEANS 

DELAY 

.0738 

.0187 

.0988 

.0637  FTask  * 42.6083** 

t nnnn 

NO 

DELAY 

.0660 

.0126 

.0865 

lQv>K  « uuuu 

FDel ay  = 5.3550* 

,055°  PDelay  = .0392 

...  F 

COLUMN 

MEANS 

.06S9 

.0156 

.0926 

rTask  X Delay  = .4156 

PTask  X Delay  = .6701 

* STATISTICALLY  SIGNIFICANT  AT  ..05  LEVEL 

**  STATISTICALLY  SIGNIFICANT  AT  .01  LEVEL 
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Table  8.  STATISTICAL  SUMMARIES  DIRECTIONAL 
AVERAGE  VARIANCES- 1 2 SUBJECTS 


DISPLACEMENT  (DRP) 


LEFT 

CENTER 

RIGHT 

ROW 

MEANS 

DELAY 

.0248 

.0017 

.0219 

.0161  FTask  = 8.4328** 

- nn?? 

NO 

DELAY 

. 0 ’ 97 

.0045 

.0207 

1 db  l\  i UvbL 

FDelay  = .3393 

*0150  PDelay  = .5775 

COLUMN 

MEANS 

.0222 

.0031 

.0213 

rTask  X Delay  = 1.2694 

PTask  X Delay  = .3007 

FO! 

LEFT 

RCE  {FRP 
CENTER 

A) 

RIGHT 

ROW 

MEANS 

DELAY 

4.5721 

1.1846 

6.5518 

4.1028  FTask  = 8.0062** 

PtjpI  - nnoo 

NO 

DELAY 

3.3319 

2.5284 

7.6548 

FDelay  = .5977 

4-5055  PDelay  >4613 

F 

COLUMN 

MEANS 

3.9520 

1.8565 

7.1033 

rTask  X Delay  * 1.0052 
PTask  X Delay  = , .3839 

* STATISTICALLY  SIGNIFICANT  AT  .05  LEVEL 

**  STATISTICALLY  SIGNIFICANT  AT  .01  LEVEL 
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Tasks  for  six  control  parameters)  did  the  non-delayed  variance  exceed  the 
delayed  variance.  Of  these,  three  were  for  the  rudder  control  variables 
(Center  Task,  rudder  pedal  deflection  (DRP)  and  Center  and  Right  Tasks, 
rudder  control  force  (FRPA)). 

There  is  some  chance  that  the  differences  in  average  variance  observed 
between  the  different  conditions  are  due  to  chance  or  happenstance.  One 
accepted  method  for  determining  the  probability  that  the  observed  phenomena 
are  due  to  chance  is  known  as  the  multivariate  analysis  of  variance.  There- 
fore, a multivariate  analysis  of  variance  in  the  average  variances  of  the 
six  control  parameters  was  performed  to  determine  statistical  significance 
of  the  differences  in  average  variance  of  the  control  parameters  lue  to 
both  the  task  and  the  delay  factors.  For  our  purposes,  two  levels  of 
statistical  significance  arg  considered  and  are  defined  to  be  those  situa- 
tions in  which  the  F ratios  resulting  from  delay  effects  being  due  to 
chance,  are  less  than  .05  or  less  than  .01. 

The  analysis  of  variance  program  was  obtained  from  what  is  known  as 
VUL2,  the  Vanderbilt  Statistical  Package  , written  by  Dr.  Laird  W.  Heal. 

The  program,  called  ANOVABS  (figure  14,  part  2),  calculates  an  analysis  of 
variance  with  two  "within"  factors,  or  repeated  measures. 

The  differences  in  variances  of  the  control  inputs  (DSS,  FSSA,  DSA, 

FSAA,  DRP,  FRPA)  for  the  two  basic  conditions  of  Delayed  and  Non-Delayed 
visual  presentation  are  shown  in  figures  15,  16,  and  17.  The  results  of 
the  multivariate  analysis  on  the  differences  are  presented  in  tables  6,  7, 
and  8.  The  F ratios  for  the  task  origins  are  statistically  significant  for 
all  tasks.  This  indicates  that  all  of  the  observed  control  parameters  were 
exercised  differently  for  each  task.  This  is  not  surprising  since  the 
tasks  are  all  different.  The  center  task  required  the  fewest  control  mani- 
pulations of  the  three  tasks.  The  principal  difference  in  left  and  right 
task  was  the  addition  of  the  turbulent  air  variable  to  the  right  task.  The 
F ratio  based  on  the  differences  of  variances  due  to  delayed  or  non-delayed 
visual  presentation  for  the  lateral  control  parameter  is  statistically 
significant  at  P = .0083  for  the  lateral  control  deflection  and  at  P = .0392 
for  the  lateral  control  force. 

FOURIER  ANALYSIS  OF  CONTROL  INPUTS.  The  question  "If  pilots  do  perform  their 
skills  differently  when  visual  stimuli  are  delayed  100  ms,  in  what  way(s) 
is  their  performance  different?"  is  difficult  to  answer  by  examining  the 
time  histories  of  the  pilot's  control  activity.  One  time  history  appears 


^Mendenhall,  William,  "Introduction  to  Probability  and  Statistics",  Third 
Edition,  Duxbury  Press,  pp.  243f. 

7 

Heal,  Laird  W. , "VUL2  Vanderbilt  Statistical  Package",  Xerox  Computer 
Users'  Group  Exchange  Program  Library,  Catalog  No.  890400-11B00. 
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much  the  same  as  another  and  in  particular,  the  time  histories  for  the 
Delayed  and  Non-Delayed  cases  also  appear  to  he  very  similar.  The  results 
of  the  multivariate  analysis  of  variance  in  Control  Forces  and  Displacements, 
indicate  that  clear  differences  exist  in  the  variances  of  the  various  control 
parameters,  but  not  what  the  nature  of  those  differences  might  be.  One  method 
of  examining  time  histories  is  to  investigate  their  frequency  content.  The 
time  histories  were  mapped  into  the  frequency  domain  to  better  evaluate  the 
exact  nature  of  the  differences  which  occur  in  piloting  technique  when  the 
pilot  subject's  visual  stimuli  have  been  delayed.  The  Fourier  transformation 
to  the  frequency  domain  was  accomplished  by  using  a published  program  package. 

The  Discrete  Fast  Fourier  Transform  (DFFT)  is  one  convenient  tool  for 
performing  the  required  mapping  from  the  time  domain  into  the  frequency  do- 
main. One  computer  program,  FOURT,  processes  the  Cooley-Tukey  Fast  Fourier 
Transform  as  defined  by: 


N-l 

-i2  7T  mn 

0 < n < N-l 

*n 

- $ 
m=o 

X e 
m 

N 

Where: 

i 

= 

imaginary 

m 

= 

summation 

index  on  the  number  of  data 

points 

n 

= 

harmonic 

N 

= 

number  of 

data  points  in  the  recorded 

time  history 

X 

m 

- 

m th  value  of  the  untransformed  data 

*n 

- 

amplitude  of  the  n th  harmonic  of  the 

transform 

An 

error 

analysis  of  this  program  appears  in  a related 

publication. 

The  various  time  histories  were  of  differing  length  making  comparisons 
of  the  results  of  the  Fourier  processing  difficult.  The  different  lengths 
were  all  augmented  with  zeros  to  make  their  lengths  1024  data  points, 

(figure  14,  page  44,  Augment)  allowing  faster  program  execution  and  a common- 
ality of  fundamental  frequencies  of  the  Discrete  Fourier  Transform  (DFT). 

So  as  not  to  introduce  major  harmonic  content  into  the  DFT's,  the  average 
value  of  each  time  history  was  removed  before  augmentation  of  the  data 
strings . 


8 

Brenner,  N.  M. , "Three  Fortran  Programs  that  Perform  the  Cooley-Tukey 
Fourier  Transformation,"  MIT  Lincoln  Laboratory  Publication  AD  657019, 

28  July  1967. 

9 

Ferris,  James  F.,  and  Nuttall,  Albert  H.,  "Comparison  of  Four  Fast  Fourier 
Transform  Algorithms,"  NUSC  Report  No.  4113,  3 June  1971,  Naval  Underwater 
Systems  Center,  Newport,  R.I. 
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The  choice  of  data  string  size  and  the  SO  ms  sampling  period  results  in  a 
fundamental  frequency  of'  .0195  Hz  per  frequency  cell.  Since  preliminary 
analysis  of  selected  time  histories  of  each  control  parameter  representing 
each  task  indicated  no  appreciable  energy  in  the  spectra  at  frequencies 
above  4 Hz,  the  calculations  were  halted  at  200  harmonics. 

The  following  spectra  were  calculated  for  each  control  parameter. 

a.  For  each  pilot  subject,  the  spectra  for  five  successful  approaches 
for  each  task  in  each  delay  condition  were  averaged. (Figure  14,  page  45, 
Average  1.)  This  produced  three  (one  for  each  task)  spectra  for  each  delay 
condition  for  each  pilot  subject. 

b.  Each  of  the  six  spectra  thus  produced  per  pilot  subject  were  then 
averaged  over  all  the  pilot  subjects,  providing  six  spectra  for  the  entire 
group  of  pilot  subjects,  one  for  each  task  for  each  delay  condition  (figure 
14,  page  45,  Average  2) . 

Thus,  thirty-six  spectra  weie  prepared  for  the  entire  group  of  pilot 
subjects;  three  (Tasks)  x two  (Delay  Conditions)  x six  (Control  Input 
Parameters).  These  are  presented  in  figure  18.  Figure  18  also  displays 
the  differences  in  the  spectra  discussed  above,  i.e.,  the  differences  in 
the  spectra  for  the  Delayed  visual  presentation  condition  minus  the  Non- 
Delayed  visual  presentation  condition.  These  differences  were  computed 
by  subtracting  the  real  and  imaginary  amplitudes  for  each  frequency  cell 
of  the  delayed  spectrum  from  the  real  and  imaginary  amplitudes  of  the 
same  frequency  cell  of  the  Non-Delayed  spectrum.  Figure  18  shows  the 
general  trend  of  the  results  of  pilot  activity  in  the  frequency  domain. 
Notice  that  the  control  input  spectra  have  decreasing  amplitude  with 
increasing  frequency  and  that  the  difference  spectra  (Delay  spectra  minus 
the  No-Delay  spectra)  have  the  same  general  trend.  Thi:.  suggests  that 
the  delay  effects  (as  indicated  by  the  difference  spectra)  are  functions 
of  frequency  and  that  the  effects  are  greater  at  around  .6  Hz.  The  results 
of  the  frequency  analysis  are  summarized  in  table  9.  The  principle  fre- 
quency and  approximate  amplitude  refer  to  the  difference  spectra  of  the 
delayed  condition  minus  the  non-delayed  condition.  The  control  input 
limit  refers  to  either  the  delayed  or  non-delayed  case  and  merely  states 
the  approximate  upper  frequency  limit  of  information  content. 


^The  spectrum  averaging  discussed  herein  is  the  arithmetic  mean  of  the 
contents  of  each  frequency  cell  (multiple  of  the  fundamental  frequency). 
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Figure  18.  Fast  Fourier  Transforms  of  Control  Inputs 
by  Task 
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Figure  18.  Fast  Fourier  Transforms  of  Control  Inputs 
by  Task  (CONT) 
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Table  9.  SUMMARY  OF  ANALYSIS  OF  FAST  FOURIER  PROCESSING 


CONTROL 

PARAMETER 

LEFT 

TASK 

CENTER 

RIGHT 

DSS 

Principle  Frequencies  (Ha) 

.2  to  .4 

.1 

.1,  .4 

Approx.  Amplitudes 

.08 

.16 

.07 

Control  Input  Limits  (Ha) 

.8 

.8 

.8 

FSSA 

Principle  Frequencies  (Ha) 

.4 

.2  to  .1,  Peak  .6 

.4 

Approx.  Amplitudes 

.05 

.04 

. 05 

Control  Input  Limits  (Ha) 

1.6 

2.4 

2.4 

3SA 

Principle  Frequencies  (Ha) 

.1 

.1  to  .6 

.2 

Approx.  Amplitudes 

.1 

.025 

.1 

Control  Input  Limits  (Ha) 

.8 

1.2 

.8 

FSAA 

Principle  Frequencies  (Ha) 

.1 

.2,  .4 

.2 

Approx.  Amplitudes 

.02 

.006 

.02 

Control  Input  Limits  (Ha) 

.8 

.8 

.8 

DRP 

Principle  Frequencies  (Ha) 

.1 

.1 

.1  to  .4 

Approx.  Amplitudes 

.015 

.006 

.006 

Control  Input  Limits  (Ha) 

1.6 

1.6 

1.6 

FRPA 

Principle  Frequencies  (Ha) 

.4 

.4,  1.2,  1.8 

.4,  .9 

Approx.  Amplitudes 

.1 

.04 

.12 

Control  Input  Limits  (Ha) 

2.4 

3.4 

3.4 
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CONCLUSIONS  AND  DISCUSSION 

The  first  question  posed  in  the  statement  of  the  problem  "Does  100  ms 
delay  of  a visual  presentation  affect  pilot  learning  performance?"  was 
answered  by  Experiment  1.  No  statistically  significant  differences  were 
found  between  the  "trials-to-criterion"  (three  successive  traps)  in  the 
Delayed  condition  and  in  the  Non-Delayed  condition. 

The  second  question  posed  in  the  statement  of  the  problem  "Do  pilots 
perform  their  piloting  skills  differently  when  their  visual  stimuli  have 
been  delayed  for  100  ms?"  has  been  answered  in  the  affirmative  insofar  as 
the  pilot  control  inputs  in  the  lateral  control  parameters  (displacement 
and  force)  are  concerned.  The  effect  of  delay  was  found  to  be  statistically 
significant  at  the  .0083  level  for  aileron  control  displacement  (DSA)  and 
at  the  .0392  level  for  aileron  control  force  (FSAA).  The  effect  of  delay 
on  the  remaining  four  control  parameters  (DSS,  FSSA,  DRP,  FRPA)  was  found 
to  be  not  statistically  significant. 

While  the  differences  in  the  mean  scores  for  all  tasks  for  the  remain- 
ing four  pilot  control  input  parameters  for  the  Delay  compared  with  the  No- 
Delay  condition  were  all  statistically  not  significant,  it  is  interesting 
to  note  that  of  the  eighteen  mean  comparisons  made  (see  tables  6,  7,  and  8), 
four  average  variance  values  were  less  for  the  Delay  condition  than  for  the 
No-Delay  condition.  (They  were  elevator  control  force  (FSSA)  during  the 
Right  Task,  rudder  pedal  deflection  (DRP)  during  the  Center  Task,  and  rudder 
pedal  force  (FRPA)  during  the  Right  and  Center  Tasks.) 

It  is  believed  thac  these  four  average  variance  values  can  be  explained. 
Three  of  the  four  comparisons  involved  rudder  control  force  and/or  deflec- 
tion. Several  approaches  by  subject  pilots  were  made  with  high  angle  of 
attack,  sufficient  to  activate  the  rudder  pedal  stall  warning  shaker.  It 
is  believed  that  the  directional  displacements  and  forces  recorded  due  to 
the  shaker  masked  the  effect  of  the  delay  condition  on  pilot  subject  induced 
control  displacements  and  forces.  The  fourth  comparison,  elevator  control 
force  during  the  Right  Task,  is  believed  to  be  similarly  masked  by  the 
rough  air  turbulence  used  in  this  task.  None  of  the  other  tasks  utilized 
rough  air  turbulence. 

The  third  question,  * If  pilots  do  perform  their  skills  differently 
when  visual  stimuli  are  delayed  100  ms,  in  what  way  is  their  performance 
different?"  has  been  resolved  by  transforming  the  pilot  control  inputs  to 
the  frequency  domain  and  comparing  the  frequency  spectra  of  the  control 
inputs  for  the  delayed  visual  presentation  to  the  spectra  for  the  non- 
delayed  visual  presentation  case.  These  comparisons  are  summarized  in 
table  S. 

The  time  histories  of  each  control  parameter  for  all  successful 
approaches  were  transformed  to  the  frequency  domain  using  the  discrete 
Fourier  transform.  The  transformations  were  averaged  for  each  given  task 
and  each  delay  condition  over  all  pilot  subjects.  The  difference  spectra 
were  formed  by  subtracting  the  average  delayed  spectrum  from  the  average 
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non-delayed  spectrum  for  each  task  and  each  control  parameter,  (figure  18). 
The  difference  spectra  show  the  effect  of  delay  to  decrease  with  increasing 
frequency.  The  major  difference  between  the  Delayed  and  No-Delayed  spectra 
typically  occurred  in  the  range  0 to  2 Hz. 

The  results  of  these  experiments  are  applicable  to  a high  performance 
simulation  (F-4)  using  a narrow  field  of  view  visual  presentation.  However, 
caution  should  be  exercised  before  any  attempt  is  made  to  extrapolate  the 
results  to  visual  systems  with  wider  fields  of  view  or  to  aircraft  having 
different  frequency  modes  such  as  large  bomber  or  transport  aircraft. 

In  conclusion,  it  has  been  determined  that  learning  performance  of 
pilot  subjects,  executing  the  tasks  specified  for  Experiment  1 and  in  the 
simulator  system  utilized,  was  not  affected  by  100  ms  delay  in  visual 
stimuli.  Perhaps  this  result  could  be  due  to  pilot  subjects  responding, 
with  extra  effort,  to  the  delayed  task  conditions,  i.e.,  they  may  have 
"tried  harder."  It  was  determined  that,  in  general,  pilot  subjects 
manipulated  their  flight  controls  differently  both  in  displacements  and 
in  control  force  when  their  visual  stimuli  were  delayed  100  ms.  These 
differences  are  indicated  both  by  the  general  trend  toward  a greater 
variance  in  control  activity  (in  some  cases  the  differences  were  statis- 
tically significant)  and  by  the  differences  in  the  frequency  spectra  for 
the  Delayed  and  Non-Delayed  conditions. 
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SECTION  V 
RECONMENDATIONS 


In  view  of  the  findings  of  Experiments  1 and  2,  the  following  studies 
are  recommended: 

a.  A similar  study  be  conducted  which  would  allow  both  a variable 

time  delay  and  variable  task  as  independent  arguments.  Sample  areas  of 
interest  would  include:  learning  performance  and  input  control  variance 

aS  functions  of  length  of  delay  time  and  task  type.  Since  the  present 
experiments  considered  only  the  carrier  landing  task,  other  task  types 
might  be  aerial  refueling,  air-to-ground  weapon  delivery,  and  formation 
flying. 

b.  A similar  study  should  be  conducted  for  a large  field  of  view 
visual  presentation  system. 

c.  A similar  study  should  be  conducted  for  large  multi-engine 
transport  type  aircraft  whose  natural  frequencies  are  vastly  different 
than  the  strike  type  of  aircraft  (the  F-4)  used  in  these  studies. 

d.  The  study  should  be  repeated  utilizing  predictive  filters 
designed  based  upon  the  frequency  spectra  of  the  differences  in  the 
delayed  and  the  non-delayed  pilot  control  input  performance.  The  pre- 
diction could  be  expected  to  reduce  the  effects  of  the  delayed  visual 
presentation. 


